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PICTURE SIGNAL TRANSMITTING 
METHOD AND APPARATUS 

RELATED APPLICAHON 

This is a continuation-in-part of U.S. patent appKcation 
Ser, No, 08/219,472, filed Mar. 29, 1994, now U.S. Pat No. 
5,473380, 

BACKGROUND OF THE INVENTION 
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The present invention relates to coding and decoding of a 
picture s^nal for transmission, and, more particularly, is 
directed to matching the type of predictive coding applied to 
pictures of the picture signal. 

In, for example, a teleconferencing system or a video 15 
telephone system, moving picture signals are compressed 
and encoded by taking advantage of intra-frame and inter- 
frame correlation so that they can be more efSciently trans- 
mitted over a communication diannel to a remote location. 

Intra-frame correlation can be utilized by an orthogonal 20 
transformation, such as a discrete cosine transformation 
(DCT). 

Inter-frame correlation can be utilized by predictive 
encoding between successive pictures. As used herein, a 
picture gena:aliy refers to an image represented by a frame. 25 
When the fields of a frame are coded in a non-interlaced 
manner, that is, separately, each field may be referred to as 
a picture. 

As shown in FIG. lA, for example, frame pictures PCI, 
PC2 and PCS are generated at time pomts tl, t2 and t3. As 
shown by shading in FIG. IB, the difference between the 
frame pictures PCI and PC2 is obtained as difference picture 
data PC12, and the difference between the frame pictures 
PC2 and PC3 is obtained as difference picture data PC23. 
Since there is a fairly small change between signals of 
temporally neighboring frames, transmission of only the 
difference picture data utilizes the transmission channel 
more efSdentiy than transmission of the original pictures. 
That is, using the difference picture data as encoded picture 
signals reduces the amount of data to be transmitted, 

However, if only the difference signals are transmitted, 
the original picture cannot be restored It is advantageous to 
occasionally transmit a picture which is not predictively 
encoded as a reference for difference picttire data, and 
because it is sometimes more efficient than transmitting the 
picture as a predictively encoded picture. 

Pictures which are encoded utiHzmg only intra-frame 
correlation and not inta:-frame correlation, are referred to 
herein as intra-pictures or I-pictures, 

Pictures which are encoded with predictive encoding 
relative to one previously encoded picture are referred to 
herein as predictive pictures or P-pictures. The previously 
encoded picture may be an I-picture or a P-picture, and 
temporally succeeds the P-picture, 

Pictures whidi are encoded with predictive encoding 
relative to at most two pictures, a temporally preceding and 
a temporally succeeding pictoe, are referred to herein as 
bi-directionaUy predictive coded pictures or B-pictures. The 
two pictures may each be an I-picture or a P-picture, When 50 
both are used, the mean value of the two pictures is obtained 
and used as a reference picture for the picture to be encoded. 

A series of pictures may be considered as groups of 
pictures having a predetermined number of frames such as 
Fl . , , F17. The luminance and chrominance picture signals 65 
of the leading frame Fl are encoded as an I-picture, the 
pidure signals of the second frame F2 are encoded as a 



B-picture, and the picture signals of the third frame F3 are 
encoded as a P-picture. The fourth and the following frames 
F4 to F17 are encoded alternately as B-pictures and 
P-pid;iires. FIG. 2A shows the reference pictures used for 
5 encoding P-pictures, while FIG. 2B shows the reference 
pictures used for encoding B-pictures. 

As shown in FIGS. 3A and 3B, there are four methods for 
encoding the maoro-blocks (discussed below) of a picture. 
When multiple methods are suitable, the method which will 
give the smallest amount of encoded data is employed on a 
macro-block by macro-block basis. Blocks Fl to F5 in FIG. 
3A represent data for frames of moving picture signals, 
whereas blocks FIX to F5X in FIG, 3B represent data for 
encoded frames. The solid line arrows in FIG. 3A show the 
15 frames to which motion vectors xl , . . x6 relate. 

The first method, shown as SPl, is to not use predictive 
encoding, that is, to use only intra-frame correlation. This is 
suitable for any macro-blocks of an I-picture, a P-picture and 
a B-picture. In other words, if less encoded data is produced 
without predictive encoding, then this method is selected 

The second metiiod, shown as SP2, is to predictively 
encode relative to a picture which temporally succeeds the 
current picture, referred to as backward predictive encoding. 
^ The ted method, shown as SP3, is to predictively encode 
relative to a picture which temporally precedes the current 
picture, referred to as forward predictive encoding. The 
second method is suitable for macro-blocks of only 
B-pictures. The third method is suitable for macro-blocks of 
P-pictures and B-pictures. 

The fourth method, shown as SP4, is to predictively 
encode relative to the mean value of two pictures, one 
temporally preceding and one temporally succeeding the 
current picture. This method is suitable for macro-blocks of 
35 only B-pictures. 

The encoding sequence will now be described. 

The first frame Fl is encoded as an I-picture using the first 
method SPl so that it is directly transmitted over a trans- 
mission channel as encoded data FIX. 

The third frame F3 is encoded as a P-picture. When the 
third method SP3, forward predictive coding, is used for a 
macro-block, difference signals from the temporally preced- 
ing frame Fl used as the reference picture, as indicated by 
a broken-line arrow SP3, and a motion vector x3 between 
the reference picture Fl and the current picture F3, are 
calculated and encoded as data F3X for that macro-block. 
Alternatively, in this or another macro-block of the P picture, 
if a smaller amount of encoded data is produced for a 
macro-block of the P picture being encoded, the first method 
SPl can be used wherein the data of the original frame F3 
are directly utilized as the transmission data F3X for that 
macro-block. 

The second frame F2 is encoded as a B-picture, 
55 When the fourth method SP4 is used to encode a macro- 
block of the frame F2, a difference between the mean value 
of the temporally preceding frame Fl and the temporally 
succeeding frame F3 is calculated, on a pixel by pixel basis. 
The difference data and the motion vectors xl and x2 are 
^ encoded as data F2X. 

When the first processing method SPl is used to encode 
a macro-block of the frame F2, the data of the original frame 
F2 forms the encoded data F2X. 

When one of the second or third methods SP2, SP3 is used 
65 to encode a macro-block of the frame F2, one of the 
difference between the temporally succeeding frame F3 and 
the current frame F2, and the difference between the tem- 



porally preceding frame Fl and the current frame E2 is 
calcukted. The difference data and one of the motion vectors 
xl, x2 are encoded as the data F2X. 

The frame F4 for the B-picture and the frame F5 for the 
P-picture are processed in a similar manner as described 5 
above to generate transmitted data F4X and F5X. 

HG. 4 illustrates an arrangement for encoding and decod- 
ing moving picture signals in accordance with the above- 
described predictive encoding scheme. As shown in FIG. 4, 
an encoding device 1 encodes input picture signals and 
transmits the encoded signals to a recording medium 3 as a 
transmission channel for recording. A decoding device 2 
reproduces the signals recorded on the recording medium 3 
and decodes these as output signals. 

The encoding device 1 includes an input terminal 10, a 
p5:e-processing circuit 11, A/D converters 12 and 13, a frame 
memory 14 including a luminance signal frame memory 15 
and a color difference signal frame memory 16, a format 
converting circuit 17 and an encoder 18. 

20 

Input terminal 10 is adapted to receive a video signal VD 
and to supply the signal VD to pre-processing circuit 11 
which functions to separate the video signal VD into lumi- 
nance signals and color signals, herein chrominance or color 
difference signals, that are applied to analog-to-digital (A/D) ^5 
converters 12 and 13, respectively. The video signals, digi- 
tized by analog- to-digital conversion by the A/D converters 
12 and 13, are supplied to frame memory 14 having memo- 
ries 15, 16 which function to store the luminance signals and 
the color difference signals, respectively, and to read out the 
signals stored therein to format converting circuit 17. 

The converts 17 is operative to convert frame format 
signals stored in the frame memory section 14 into Mock 
format signals. As shown in HG. 5A, pictures are stored in 
the frame memory section 14 as frame-format data having V 
lines each consisting of H dots. The converting circuit 17 
divides each frame into N slices, each slice comprising a 
multiple of 16 lines. As shown, in FIG. 5B, the converter 17 
divides each slice into M maoro-blocks. As shown in HG. 
5C, each macro-block represents luminance signals Y cor- ^ 
responding to 16x16 pixels or dots, and associated chromi- 
nance Cr, Cb signals. These luminance signals are sub- 
divided into blocks Yl to Y4, each consisting of 8x8 dots. 
The 16x16 dot luminance signals are associated with 8x8 
dot Cb signals and 8x8 dot Cr signals. The converter 17 is 45 
also operative to supply the block format signals to the 
encoder 18, which is described in detail below with refer- 
ence to FIG. 6. 

The encoder 18 operates to encode the block format 
signals and to supply the encoded signals as a bitstream ova* 50 
a transmission channel for recording on the recording 
medium 3. 

The decoding device 2 includes a decoder 31, a format 
converting circuit 32, a frame memory section 33 including 
a luminance signal frame memory 34 and a color difference 55 
signal frame memory 35, digital-to-analog converters 36 and 
37, a post-processing drcuit 38 and an output terminal 30. 

The decoder 31 is operative to reproduce encoded data 
from the recording medium 3 and to decode the encoded 
data, as described in detail below with reference to FIG, 9, 60 
and to supply decoded data signals to format converting 
ckcuit 32 which is operative to convert the decoded data 
signals into frame format data signals and to supply the 
frame format data s%nals as luminance signals and color 
difference signals to the memory 33. The memories 34, 35 65 
of the memory 33 function to store die luminance and 
chrominance signals, respectively, and to ^piy these signals 



to D/A converters 36 and 37, respectively. The analog 
s^nals from converters 36, 37 are synthesized by a post- 
processing circuit 38 which functions to form output picture 
signals and to output them to output terminal 30, and thence 
5 to a display unit, such as a CRT, not shown, for display. 

FIG. 6 illustrates the encoder 18 shown in FIG. 4, 

Generally, the encoder 18 stores three pictures, the current 
picture and the pictures tenq)orally preceding and succeed- 
ing the current picture. Based on the sequential position of 
the current picture in the group of pictures, the picture 
coding type (I, P or B) is selected for each picture. The 
picture type sequence is determined by a user using picture 
type input device 65, independent of the pictures ^plied to 
an input terminal 49. 

The encoder 18 also chooses one of frame-based and 
field-based predictive encoding as will be explained witii 
reference to FIG. 7, and further chooses one of frame-based 
and field-based DCT encoding as will be explained with 
reference to FIG. 8. For eadi picture, appropriate motion 
vectors are obtained and the picture is predictively encoded 
relative to zero, one or two previously encoded pictures 
which have been locally decoded and which are referred to 
as reference pictures to form a difference data signal. The 
difference data signal is orthogonally transformed into 
blocks of coefficient data which are quantized, variable 
length encoded and transmitted as encoded data. 

At the encoder 18, the quantized data are dequantized, 
inverse orthogonally transformed, and stored as the refer- 
ence pictures. Hie predictive encoding applies the motion 
vector(s) obtained for the current picture to the reference 
picture(s) to produce a prediction picture which is subtracted 
from the current picture to yield the difference data. 

The elements of the encoder 18 will now be explained in 
35 detail. 

Picture data for encoding is supplied macro-block by 
macro-block to the input terminal 49 and tiience to a motion 
vector detection drcuit 50 which is operative to process the 
picture data of respective frames as I-pictures, P-pictures or 

40 as B-pictures, in accordance with a predetermined sequence 
for each group of pictures, as shown for example, in FIGS. 
2A, 2B. The circuit 50 applies the picture data of the current 
frame to a frame memory 51 having frame memories 51a, 
51^, Sic used for storing a temporally preceding picture, the 

45 current picture and a temporally succeeding picture, respec- 
tively. 

More specifically, the frames Fl, F2, F3 are stored in the 
memories 51a, Sib, 51c, respectively. Then the picture 
stored in memory 51c is transferred to memory 51a. The 
50 frames F4, F5 are stored in the memories 51fe, 51c, respec- 
tively. The operations of transferring the picture in memory 
51c to memory 51a and storing the next two pictures in 
memories 515, 51c are repeated for the remaining pictures in 
the group of pictures. 

55 After the pictures are read into the memory and ten:^)o- 
rarily stored, they are read out and supplied to a prediction 
mode changeover circuit 52 which is ad^ted to process the 
current picture for one of frame based and field based 
predictive encoding. After processing the first frame picture 

60 data in a group of pictures as an I-picture and before 
processing the second frame picture as a B-picture, the 
motion vector detection drcuit 50 processes the third frame 
P-picture. The processing sequence is different from the 
sequence in which the pid;ures are supplied because the 

65 B-picture may involve backward prediction, so subsequent 
decoding may require that the P-picture temporally succeed- 
ing the B-picture have been previously decoded. 
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The motion vector detection circuit 50 calculates as an 
estimated value for intra-coding for each macro-block, the 
sum of absolute values of prediction errors for the frame 
prediction mode for each macro-block and the sum of 
absolute values of prediction errors for the field prediction 5 
mode for each macro-block and supplies these sums to the 
prediction decision circuit 54 which compares these sums 
and selects frame prediction mode or field prediction mode 
in accordance with the smallest of these values and provides 
the selected mode to the prediction mode diangeover circuit lo 
52. 

If the frame prediction mode is selected, the prediction 
mode changeover circuit 52 outputs the four luminance 
blocks Yl to Y4 and the two chrominance or color difference 
blocks Cb, Cr of each macro-blockreceived from the motion ^ ^ 
vector detection circuit 50 without processing. As shown in 
FIG. 7A, odd or first field line data, indicated by soUd lines, 
and even or second field line data, indicated by dashed lines, 
alternate in each luminance and color difference block as 
received from the motion vector detection circuit 50. In FIG. 20 
7A, a indicates units for motion compensation. In the frame 
prediction mode, motion compensation is performed with 
four luminance blocks (macro-blocks) as a unit and a single 
motion vector is associated with the four luminance blocks 
Yl to Y4. 25 

If the field prediction mode is selected, the prediction 
mode changeover circuit 52 processes the signals received 
from the motion vector detection circuit 50 so that each of 
the four luminance blocks comprises data from a single field 
and the two color difference blocks have non-interlaced odd 
and even field data. Spedficaliy, as shown in FIG. 7B, the 
luminance blocks Yl and Y2 have odd-field data and the 
luminance blocks Y3 and Y4 have even-field data, while the 
uppa: halves of the color difference blocks Cb, Cr represent 
odd field color difference data for the luminance blocks Yl 
and Y2 and fiie lower halves of the color diffa-ence blocks 
Cb, Cr represent even field color difference data for the 
luminance blocks Y3 and Y4. In FIG. 7B, b indicates units 
for motion compensation. In the field prediction mode, 
motion condensation is performed separately for the odd- ^ 
field blocks and even-field blocks so that one motion vector 
is associated with the two luminance blocks Yl and Y2 and 
another motion vector is associated with the two luminance 
blocks Y3 and Y4. 

45 

The prediction mode changeover circuit 52 supplies the 
current picture, as processed for frame based or field based 
predictive encoding, to arithmetic unit 53 of FIG. 6. The 
arithmetic unit 53 functions to perform one of intra-picture 
prediction, forward prediction, backward prediction or 
bi-directional prediction. A prediction decision circuit 54 is 
adapted to select the best type of prediction in dependence 
upon the prediction error signals associated with the current 
picture signals. 

The motion vector detection circuit 50 calculates, for the 55 
current picture, the sum of absolute values of the differences 
between each Aij and the average value of the Aij in each 
macro-block ZLAij-(average of Aij)i and supplies the sum as 
an estimated value for intra-coding to the prediction decision 
circuit 54. ^ 

The motion vector detection circuit 50 calculates the sum 
of absolute values (or sum of squares) of the difference (Aij 
-Bij) between signals Aij of the macro-blocks of the current 
picture, and signals Bij of the macro-blocks of the prediction 
picture ElAij-Bijl in each of firame prediction mode and field 65 
prediction mode. As explained above, the motion vector(s) 
for the current picture are applied to the reference picture(s) 
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to generate the prediction picture. When the reference pic- 
ture temporally precedes the current picture, the quantity 
ZlAij-Biji is referred to as a forward prediction error signal, 
and when the reference picture temporally succeeds the 

5 cuirent picture, the quantity ElAij-Bijl is referred to as a 
backward prediction error signal. When the prediction pic- 
ture is the mean of a ten^)oraily preceding and a temporally 
succeeding reference picture, as motion-compensated, the 
quantity IIAij-Bijl is referred to as a bi-directional predic- 

10 tion error signal. 

The circuit 50 supplies the forward frame prediction, the 
forward field prediction, the backward frame prediction, the 
backward field prediction, the bi-directional frame predic- 
tion and the bi-directional field prediction error signals to the 
15 prediction decision circuit 54. 

The prediction decision circuit 54 selects one of intra- 
coding, forward inter-picture prediction, backward inter- 
picture prediction or bi-directional inter-picture prediction 
and one of frame and field prediction mode in accordance 
with the smallest of the estimated value for intra-coding and 
the forward frame, the forward field, the backward frame, 
the backward field, the bi-directional frame and the 
bi-directional field prediction error signals. The arithmetic 
unit 53 predictively encodes the current picture, as processed 
by the frame or field changeover circuit 52, in accordance 
with the prediction mode selected by the prediction decision 
circuit 54. 

The motion vector detection circuit 5€ serves to calculate 
and supply the motion vector(s) associated witii the selected 
prediction mode to a variable length encoding circuit 58 and 
a motion compensation circuit 64, explained later. 

The sums of the absolute values of the inter-frame dif- 
ferences (prediction errors) on the macro-block basis are 
35 supplied from the motion vector detection circuit 50 to the 
prediction mode changeover circuit 52 and to the prediction 
decision circuit 54, in the manner as described above. 

The arithmetic unit 53 supplies predictively encoded data, 
also referred to as difference data, for the current pidure to 
40 a DCr mode changeover circuit 55 which is adapted to 
process the current picture for one of frame based and field 
based orthogonal transformation. 

The DCT changeover circuit 55 functions to compare the 
encoding eflSciency when the DCT operations for the macro- 

45 blocks in a picture are performed with the odd field data 
alternating with the even field data, that is, for frame based 
orthogonal transformation, as shown in FIG. 8A, with the 
encoding efSdency when the E>CT operations for the macro- 
blocks in a picture are performed with the odd field data 

50 separated from the even field data, that is, for field based 
orthogonal transformation, as shown in FIG. 8B. The circuit 
55 functions to select the mode with the higher encoding 
efSdency. 

To evaluate the encoding efificiency for frame based 
^ orthogonal transformation, the DCT mode changeover cir- 
cuit 55 places the luminance macro-block data into inter- 
laced form, as shown in FIG. 8A, and calculates the diffa-- 
ences between the odd field line signals and even field line 
signals vertically adjacent to each other, and finds the sum 
^ of absolute values of the differences EFM, or the sum of 
squared values of the differences. 
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J=i p=l j=l i=l 

To evaluate the encoding efficiency for field based 
ortiiogonal transformation, the DCT mode diangeover cir- 
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cuit 55 places the luminance macro-block data into non- 
interlaced fontu as shown in FIG. 8B, and calculates the 
differences between vertically adjacent odd field line signals 
and the differences between vocally adjacent even field 
line signals, and finds the sum of absolute values of the 5 
differences EFD, or the sum of squared values of the 
differences- 



16 15 Eq. 2 

EFD - Z I (K V) - o{i + 1 j)I + Hi J) - e{i + UJ) 

/=! 10 

The DCr changeover circuit 55 compares the difference 
between the frame based and field based sums of the 
absolute values with a predetermined threshold and selects 
frame based DCT transformation if the difference EFM- 
EFD is less than the predetermined threshold. 
" If the frame prediction mode is selected in the prediction 
mode changeover circuit 52, the probability is high that the 
frame DCT mode will be selected in the DCT mode 
changeover circuit 55. If the field prediction mode is 
selected in tiie prediction mode changeover circuit 52, the 
probability is high that the field DCT mode will be selected 
in the DCT mode changeover circuit 55. However, since this 
is not necessarily the case, the prediction mode changeover 
circuit 52 sets the mode which will give the least value of the 
sum of the absolute values of prediction errors, while the 
DCT mode changeover circuit 55 sets the mode which will 
give the optimum orthogonal transformation encoding effi- 
ciency. 

If frame based orthogonal transformation mode, also 
referred to as frame DCT mode, is selected, the DCT mode 
changeover circuit 55 functions to ensure that the four 
luminance blocks Yl to Y4 and two color difference blocks 
Cb, Cr represent altOTating or interlaced odd and even field 
Hnes, as shown in FIG. 8A, 

If field based orthogonal transformation mode, also 
referred to as field DCT mode, is selected, the DCT mode 
changeover circuit 55 functions to ensure that each of the 
luminance blocks represents only one field, and that each of 
the color difference blocks has segregated or non-interlaced 4^ 
odd and even field lines, as shown in FIG, 8B. 

The DCT mode changeover circuit 55 functions to output 
the data having the configuration associated with the 
selected DCT mode, and to output a DCT flag indicating the 
selected DCT mode to the variable length encoding circuit 45 
58 and the motion compensation circuit 64. 

The DCT mode changeover circuit 55 supplies ^propri- 
ately configured difference picture data to a DCT circuit 56 
shown in FIG. 6 which is optative to orthogonally trans- 
form it using a discrete cosine transformation into DCT 50 
coeflScients, and to supply the DCT coefficient data to a 
quantization circuit 57 tiiat functions to quantize the coef- 
ficient data with quantization steps selected in accordance 
with the volume of data stored in a transmission buffer 59 
and to supply quantized data to a variable length encoding 55 
circuit 58, 

The variable length encoding circuit 58 is also supplied 
with the quantization step or scale data from the quantization 
circuit 57, prediction mode data from the prediction decision 
circuit 54, that is data indicating which of tiie intrapicture 60 
prediction, forward prediction, backward prediction or 
bi-directional prediction is used, and motion vector data 
from the motion vector detection circuit 50. The encoding 
circuit 58 also receives prediction flag data from the pre- 
diction decision circuit 54 comprising a flag indicating 65 
which of the frame prediction mode or the field prediction 
mode is used, and prediction flag data from the DCT mode 
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changeover drcuit 55 comprising a flag indicating which of 
the frame DCT mode or the field DOT mode is used. This 
information is placed into the header portion of the encoded 
data stream- 

5 The variable length encoding circuit 58 serves to encode 
the quantized data and the header information using a 
variable length code such as a Huffman code, in accordance 
with the quantization step data supplied from the quantiza- 
tion circuit 57, and to output the resulting data to a trans- 

10 mission buffer 59, 

The quantized data and quantization step are also supplied 
to a dequantization circuit 60 which serves to dequantize the 
quantized data using the quantization step, and to supply the 
recovered DCT coefttcient data to an inverse DCT circuit 61 
that functions to inverse transform the DCT coefficient data 
to produce recovered difference data and to supply the 
recovered difference data to an arithmetic unit 62. 

The arithmetic unit 62 combines the recovered difference 
data with a previously encoded and decoded reference 
picture, as motion compensated, to produce decoded data for 
a reconstructed picture which will be used as a reference 
picture and which is read into one of two frame memories 
63a, 63b. The memories 63a, 63^? are adapted to read out the 
reference picture data stored therein to a motion compen- 
sation circuit 64 that uses the motion vectors from die 
motion vector detection drcuit 50 to produce a prediction 
picture from the reference picture. Specifically, the circuit 50 
uses the motion vector to alter the readout address of the 
reference picture from the memory 63a or 6^, 

For a group of pictures, after the first frame I-picture data 
and the third frame P-picture data are stored in the forward 
and backward prediction picture memories or units 63a, 63^, 
respectively, the second frame B-picture data is processed by 
22 the motion vector detection circuit 50. The prediction deci- 
sion circuit 54 selects the frame or field prediction mode, 
while setting the prediction mode to one of intra-frame 
prediction mode, forward prediction mode, backward pre- 
diction mode and bi-directional prediction mode in coire- 
^ spondence with the sum of absolute values of predictive 
errors by macro-block. 

Since a reconstructed B-picture is not used as a reference 
picture for other pictures, it is not stored in the frame 
memory 63. 

45 It will be appreciated that the frame memory 63 has its 
forward and backward prediction picture units 63a, 63b 
bank-exchanged as needed so that a picture stored in one of 
the units 63a or 63b can be outputted as either a forward or 
a backward prediction picture. 

50 The motion compensation circuit 64 functions to supply 
the motion con^ensated data as a prediction picture to die 
arithmetic unit 62 and to the arithmetic unit 53 which 
subtracts the prediction picture from die P-picture or the 
B-picture currently being predictively encoded. 

55 More specifically, when the motion vector detection cir- 
cuit 50 receives picture data for an I-picture from die 
forward original picture unit 51a, the prediction decision 
circuit 54 selects the intra-frame prediction mode and sets a 
switch 53d of die arithmetic unit 53 to an input contact a, 

50 This causes die I-picture data to be inputted directly to the 
DCT mode changeover circuit 55, In diis case, no predic^on 
picture is expected from the motion compensation circuit 64. 
The I-picture data is also supplied to the forward prediction 
picture unit 63a. 

'5 When the forward prediction mode is selected by die 
prediction decision drcuit 54, the circuit 54 also sets die 
switch 53d to an input contact b which causes the arithmetic 



unit 53^ to subtract the prediction picture, produced by the 
motion conq)ensation circuit 64, from the picture read out 
from the memory 51, for each macro-block on a pixel by 
pixel basis, to produce diffaence data. The P-picture, after 
encoding and local decoding, is supplied to one of tiie units 5 
63a, 63b. For example, if the P-picture immediately follows 
an I-picture, then the P-picture is stored in the backward 
prediction picture unit 63b. 

For forward predictive encoding, the prediction picture is 
a reference I-picture or P-picture read out from the forward 
prediction picture unit 63a of the frame memory 63 and 
motion-compensated by the motion compensation circuit 64 
in accordance with the motion vector outputted from the 
motion vector detection circuit 50. More specifically, for 
eadi macro-block, the motion compensation circuit 64 shifts 15 
the readout address of the forward prediction picture unit 
63a in an amount corresponding to the motion vector 
currently output by the motion vector detection circuit 50, 

When the backward prediction mode is selected by the 
prediction decision circuit 54, the circuit 54 also sets the 
switch 53<i to an input contact c which causes the arithmetic 
unit 53^ to subtract the prediction picture, produced by the 
motion compensation circuit 64, from the picture read out 
from the memory 51, on a pixel by pixel basis, to produce 
difference data. 

For backward predictive encoding, the prediction picture 
is a P-picture read out from the backward prediction picture 
unit 63^ of the frame memory 63 and motion-compensated 
by the motion compensation circuit 64 in accordance with 
the motion vector outputted from the motion vector detec- 
tion circuit 50. More spedficaEy, for eadi macro-block, the 
motion compensation circuit 64 shifts the readout address of 
the backward prediction picture unit 63b in an amount 
corresponding to the motion vector currently output by the 
motion vector detection circuit 50. 

When the bi-directional prediction mode is selected by die 
prediction decision circuit 54, the circuit 54 sets the switch 
53d to an input contact d which causes the arithmetic unit 
53c to subtract a prediction picture from the picture read out ^ 
from the memory 51, on a pixel by pixel basis, to produce 
difference data. The prediction picture is the mean value of 
a forward prediction picture and a backward prediction 
picture. 

In the case of bi-directional prediction, the picture stored 45 
in the forward prediction picture unit 63a, and the picture 
stored in the backward proliction picture unit 63b, are read 
out and motion-compensated by the motion compensation 
circuit 64 in dependence upon the motion vectors outputted 
from the motion vector detection circuit 50. More 50 
specifically, for each macro-block, the motion condensation 
circuit 64 shifts the readout address of the forward and 
backward prediction picture units 63a, 63^ in an amount 
corresponding to the appropriate one of the motion vectors 
currently ou^ut by the motion vector detection circuit 50. 55 

The transmission buffer 59 ten^rariiy stores the data 
supplied thereto, generates control data indicating the vol- 
ume of data stored therein and supplies the control data to 
the quantization circuit 57. When the volume of data stored 
in the transmission buffer 59 reaches a predetermined uppa* 60 
limit value, the control data from the transmission buffer 59 
causes the quantization scale of the quantization drcuit 57 to 
increase so as to decrease the volume of the quantized data. 
Similarly, when the volume of data stored in the transmis- 
sion buffer 59 reaches a predetermined lower limit value, the 65 
control data from the transmission buffer 59 causes the 
quantization scale of the quantization circuit 57 to decrease 
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so as to inorease the volume of the quantized data. In this 
manner, the transmission buffer 59 prevents the data sup- 
plied thereto from overflowing or underfiowing its capacity. 
The data stored in the transmission buffer 59 are read out at 

5 a predetermined timing to an output terminal 69 and thence 
to a transmission channel for recording on, for exan^le, the 
recording medium 3. 

Although the foregoing description has been made with 
reference mainly to the luminance blocks, the color differ- 

10 ence blocks are similarly processed and transmitted using 
the motion vector whidi corresponds to the motion vector of 
the luminance block halved in both the vertical and hori- 
zontal directions. 
FIG. 9 illustrates the decoder 31 shown in FIG. 4. 

^ ^ The reproduced encoded picture data transmitted from the 
recording medium 3 is ^plied to a reception circuit, not 
shown, or to an input terminal 80 which applies the encoded 
picture data to a reception buffer 81 that serves to terrqx)- 
rarily store the encoded picture data and to supply this data 

^ to a variable length decoding circuit 82 of a decoding circuit 
90. 

The variable length decoding circuit 82 functions to 
variable length decode the encoded data, to ou5>ut the 
recovered motion vector, prediction mode data, prediction 
flags and DCT flags to the motion compensation circuit 87, 
and to output flie quantization step data and variable length 
decoded picture-data, including the predictive mode, the 
motion vector, the predictive flag, die DCT flag and the 
quantized picture data for each macro-block, to an inverse 
quantization drcuit 83. 

The inverse quantization circuit 83 is ad^ted to dequan- 
tize the picture data supplied from the variable length 
decoding circuit 82 in accordance with the quantization step 
data supplied from the variable length decoding circuit 82 
and to output the thus recovered coefficient data to an 
inverse transformation TDCT circuit 84. 

The IDCr circuit 84 is adapted to perform an inverse 
transformation on the recovered coefficient data to produce 
recovered difference data, and to supply the recovered 
difference data to an arithmetic unit 85, 

If the recovered difforence data supplied from the IDCT 
circuit 84 represents an I-picture, the arithmetic unit 85 does 
not process the data and simply suppUes it throu^ an output 
45 tominal 91 to the format converting drcuit 32 shown in 
FIG. 4, and to a forward prediction picture unit 86a of a 
frame memory 86. 

If the recovered difference data supplied from the IDCT 
circuit 84 represents a macro-block of a P-picture produced 
50 in the forward prediction mode, then the reference picture 
data of the preceding frame, as stored in the forward 
prediction picture memory 86a of the frame memory 86, is 
read and motion-con^)ensated by a motion con[^)ensation 
circuit 87 in dependence upon the motion vector outputted 
55 from the variable length decoding circuit 82 to generate a 
prediction picture. Specifically, the motion conq)ensation 
circuit 87 uses the motion vector to alt^ the read out address 
supplied to the memory 86a. The arithmetic unit 85 adds the 
prediction picture to the recovered difference data to pro- 
60 duce a decoded or reconstructed picture which is stored in a 
backward prediction picture memory 86^ of the frame 
memory 86, The decoded P-picture is retained in the decoder 
31, and output after the next B-pid:ure is decoded and 
ou^ut, so as to restore the pictures to the order in which they 
65 were supplied to the encoder 18 of FIG, 4. 

Even if the macro-block of the P-picture was encoded as 
intra-coded mode data, the decoded P-picture is directly 
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stored in tiie backward prediction picture unit 86^, without 
being output to the output temainal 91 by the ari&metic unit 
85. 

If the recovered difference data supplied from the IDCT 
circuit 84 represents a macro-block of a B-picture encoded 5 
in the intra-coding mode, as determined from the prediction 
mode supplied from the variable length decoding circuit 82 
to the motion compensation circuit 87, a prediction picture 
is not generated. 

If the recovered difference data supplied from the IDCT 
circuit 84 represents a macro-block of a B-picture encoded 
in the forward prediction mode, as determined from the 
prediction mode supplied from the variable length decoding 
circuit 82 to the motion condensation circuit 87, the data 
stored in the forward prediction picture unit S6a of the frame 
memory 86 is read out and motion compensated by the 
motion compensation circuit 87 using the motion vector 
supplied from the variable length decoding circuit 82 to 
form the prediction picture. The arithmetic unit 85 sums the 
recovered difference data with the prediction picture to form 
the recovered B-picture. 

If the recovered difference data supplied from the IDCT 
circuit 84 represents a macro-block of a B-picture encoded 
in the backward prediction mode, as determined from the 
prediction mode supplied from the variable length decoding 
circuit 82 to the motion compensation circuit 87, the data 
stored in the backward prediction picture unit S6b is read out 
and motion compensated by the motion compensation cir- 
cuit 87 using the motion vector supplied from the variable 
length decoding circuit 82 to form the prediction picture. 
The arithmetic unit 85 sums the recovered difference data 
with the prediction picture to form the recovered B-picture. 

If the recovered difference data supplied from the IDCT 
circuit 84 represents a macro-block of a B-picture encoded ^5 
in the bi-directional prediction mode, as determined from 
the prediction mode supplied from the variable length 
decoding circuit 82 to the motion compensation circuit 87, 
the data stored in both the forward and backward prediction 
picture memories S6a, S6b are read out and respectively ^ 
motion compensated by the motion compensation circuit 87 
using the motion vectors supplied from the variable length 
decoding drcuit 82, then averaged to form the prediction 
picture. The arithmetic unit 85 sums the recovered difference 
data with the prediction picture to form the recovered 
B-picture. 

The recovered B-picture is supplied via the output termi- 
nal 91 to the format converting circuit 32. However, since 
the B-picture is not utilized for generating a prediction 
picture for other pictures, it is not stored in the frame 
memory 86. 

After ou^utting of the B-picture, picture data of the 
P-picture stored in tiie backward prediction picture unit S6b 
is read and supplied via the motion compensation circuit 87 
to the arithmetic unit 85. Motion con^nsation is not 55 
performed at this time. 

The counterpart circuits to the prediction mode 
changeover circuit 52 and the DCT mode changeover drcuit 
55 in the encoder 18 of FIG. 6 are not shown in the decoder 
31. The processing to be performed by these circuits, that is, 60 
the processing for restoring the configuration in which 
odd-field line signals and even-field line signals are sepa- 
rated from each other to the configuration in which odd and 
even-field line signals alternate with each other, is per- 
formed by the motion compensation circuit 87. 65 

The processing of the luminance signals has been 
explained in the foregoing. As will be appreciated by one of 
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ordinary skill in the art, the processing of the color differ- 
ence signals is earned out in a similar manner. However, the 
motion vector employed in such case is the motion vector for 
luminance signals which is halved in both the vertical and 
5 horizontal directions. 

FIG. 10 shows the signal to noise ratio (SNR) for pictures 
transmitted using the above-described technique. As can be 
seen, the best quality transmission is obtained for I-pictures, 
good quality transmission is obtained for P-pictures, and the 
10 poorest quality transmission is obtained for B-pictures, 
Thus, if the transmission path has adequate cq)acity, it is 
preferable to transmit a picture as an I-picture. 

If ail pictures cannot be transmitted as I-pictures, it is 
better to transmit a series of pictures as shown in FIG, 10* 
rather than in a format in which one average picture quaEty 
is used for all pictures. Hie tedmique shown in FIG. 10 
takes advantage of the human visual sense charactaistic of 
perceiving a series of changing picture quality, as shown in 
FIG, 10, as of higher quality than a series of unchanging 
picture quality, for a predetermined transmission rate. 

Accordingly, in the configuration of FIG. 6, transmission 
rate control is carried out by the quantizer 57 in order to 
attain the picture quality perceived as better. 

25 To dub pictures, two coder-decoder (codec) units are used 
in series. However, the picture quality obtained from the 
second codec is substantially worse than the picture quality 
obtained from the first codec, as explained below, 

FIG. 11 shows a configuration representing two codecs 

30 connected by an analog connection, namely, coder 201, 
decoder 202, coder 203 and decode 204, cormected in 
series. 

In FIG. 11, an analog video signal is supplied to an input 
terminal 200 as an input signal a. The input trndnal 200 

35 functions to apply the analog video signal to an A/D con- 
verter 211 of coder 201, The converter 211 is adapted to 
convert the analog video signal to a digital video signal, and 
to apply the digital video signal to coding circuit 212 that 
serves to encode this signal as previously described to 

40 produce a coded digital video signal. 

The coded digital video signal from coding circuit 212 of 
coder 201 is supplied to a decoding circuit 213 of decoder 
202 which is adapted to decode the coded digital video 
signal and to apply the decoded video signal to D/A con- 

45 verter214. 

The analog video signal produced by D/A converter of 
decoder 202 is supplied as an ou^ut signal b to the coder 
203, which functions in a similar manner as the coder 201. 

The digital video signal produced by the coder 203 is 
supplied to decoder 204 which functions in a similar manner 
as the decoder 202. The decoder 204 delivers an analog 
video signal as an output signal c to a terminal 205, which 
may transmit the signal c to anotiier coda: (not shown) and 

FIG. 12 shows the SNR of the output signals b, c shown 
in FIG. 11. The SNR of the output signal c is seen to be 
substantially worse than the SNR of the output signal b. 
The deterioration in picture quality results from a mis- 
60 match between the picture type applied in the first codec and 
the picture type applied in the second codec. Namely, if a 
picture coded as a B picture in the first coder/decoder pair is 
coded as, e.g., P picture in the second coder/decoder pair, a 
great deterioration of picture quality results because the 
65 picture quality changes as a function of the picture type. 
Since the deterioration in picture quality results from the 
mismatdi between picture types of respective stages of 
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codecs, such deterioration siimlarly takes place when digital 
connections are used between respective codecs. 

FIG. 13 shows a configuration representing two codecs 
connected by a digital connection, namely, coder 302, 
decoder 303, coder 304 and decoder 305, connected in 5 
series. 

An analog video signal is supplied to terminal 300, which 
supplies the analog video signal as an input signal a to A/D 
converter 301 that serves to digitize the signal a, and to apply 
the digital signal to a digital intrface 311 of coder 302. The 
digital interface 311 applies the signal supplied thereto to a 
coding circuit 312 which encodes or compresses tiie digital 
video data to an encoded digital video bit stream. 

The encoded digital video signal from the coding circuit 
312 is supplied to decoding circuit 313 of decoder 303 that 
decodes the signal supplied thereto, and applies the decoded 
signal to digital interface 314. The interface 314 functions to 
output the decoded signal as an ou^ut signal b. 

The output signal b is supplied to coder 304 which 20 
functions in a similar manner as coder 302 to produce a 
coded signal that is applied to decoder 305 which functions 
in a similar manner as decoder 303. The digital signal output 
from the decoder 303 is supplied to a D/A converter that 
serves to convert the signal supplied thereto to an analog 25 
video signal and to supply tiie analog video signal as an 
ou^ut signal c to ou^ut terminal 307. 

FIG. 12 also generally rq)resents the SNR of the ou^ut 
signals b, c shown in FIG. 13. 

30 

OBJECTS AND SUMMARY OF THE 
INVENTION 

Therefore, an object of the present invention is to provide 
a method and apparatus for encoding and decoding picture 
signals which avoid the aforementioned disadvantages of the 
prior art 

Another object of the present invention is to provide a 
method and apparatus for transmitting and receiving picture 
signals in serial stages which minimizes the deterioration in 
picture quality at each stage. 

Yet another object of the present invention is to match the 
type of coding applied to pictures of a picture signal in serial 
processing stages, each stage conoyprising coding and decod- 
ing. 45 

In accordance with one embodiment of the present 
invention, apparatus and method for processing a digital 
picture signal operate by receiving a digital picture signal 
which has picture type data included in a data identification 
area of the digital picture signal and which indicates one of 5Q 
intrapicture coding, predictive coding and bidlrectionaHy 
predictive coding for respective pictures represented by the 
digital picture signal. The picture signal is encoded as a 
function of the picture type data to produce an encoded 
picture signal 55 

In accordance with another embodiment of the present 
invention, apparatus and method for processing an encoded 
digital picture signal operate by decoding the encoded 
digital picture signal so as to produce picture type data 
which represents the type of encoding of the encoded digital 60 
picture signal and to produce a decoded digital picture 
signal The picture type data is added to a data identification 
area of the decoded digital picture signal to produce an 
ou^ut signal. 

The above, and other objects, features and advantages of 65 
the present invention will be apparent from the following 
detailed description of the preferred embodiments of the 
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present invention when read in conjunction with the accom- 
panying drawings in which corresponding parts are identi- 
fied by the same reference numeral. 

^ BRIEF DESCRIPTION OF THE DRAWINGS 

FIGS. lA and IB are pictures illustrating intar-frame 
correlation; 

FIGS. 2A and 2B are diagrams illustrating types of 
pictures used in predictive encoding; 
10 FIG. 3 is a diagram illustrating how picture signals are 
converted into encoded data for transmission; 

FIG. 4 is a block diagram showing a conventional device 
for encoding and decoding picture signals; 

FIG. 5 is a diagram referred to in explaining the operation 
of the format converting circuit shown in FIG. 4; 

FIG. 6 is a block diagram showing the encoder of the 
device shown in FIG. 4; 

FIG. 7 is a chart referred to in explaining tiie predictive 
20 encodmg operation of the encoder shown in FIG. 6; 

FIG. 8 is a chart referred to in explaining the orthogonal 
transformation operation of the encoder shown in FIG. 6; 

FIG. 9 is a block diagram showing the decoder of the 
device shown in FIG. 4; 
^5 FIG. 10 is a graph showing picture quality as a function 
of picture type in a transmitted signal; 

FIG. 11 is a block diagram showing two conventional 
video codecs connected in series using an analog connec- 
tion; 

30 

FIG. 12 is a graph showing picture quality of the signals 
output by the codecs of FIG. 11; 

FIG. 13 is a block diagram showing two conventional 
video codecs connected in series using a digital connection; 
35 FIG. 14 is a block diagram showing two video codecs 
according to the present invention connected in series using 
an analog connection; 

FIG. 15 is a graph showing picture quality of the signals 
ou^ut by the codecs of FIG. 14; 
40 FIG. 16 is a block diagram showing two video codecs 
according to the present invention connected in series using 
a digital connection; 

FIGS. 17A, 17B and 17C illustrate the data structure of a 
decoded digital video signal; 
45 FIGS. 18A, 18B and 18C are diagrams illustrating various 
encoding structures of groups of pictures; 

FIG. 19 is a block diagram showing a coder circuit 
according to the present invention; and 

FIG. 20 is a block diagram showing a decoding circuit 
50 according to the present invention. 

DETAILED DESCRIPTION OF THE 
PREFERRED EMBODIMENTS 

The present invention comprises a picture type identifier 
55 which is included with the picture signal at all times, namely, 
when the signal is encoded and when the signal is decoded. 
The picture type indicates one of intra-picture coding (an 
I-picture), predictive coding (a P-picture) and 
bi-directionally predictive coding (a B-picture). An I-picture 
60 includes macro-blocks encoded by only intra-coding. A 
P-picture includes macro-blocks encoded by intra-coding 
and/or macroblocks encoded by forward predictive coding. 
A B-picture includes macro-blocks encoded by intra-coding 
and/or macroblocks encoded by forward predictive coding 
65 and/or macroblocks encoded by backward predictive coding 
and/or macroblocks encoded by bi-directionally predictive 
coding. 
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Initially, each picture of a picture signal is encoded as a 
function of a respective picture type, then appropriately 
decoded. The decoded picture signal includes the respective 
picture types, preferably in the vertical blankii^ interval of 
each decoded picture. The decoded picture signal may be 5 
further processed by, for exan^le, dubbing. 

When the pictures of the decoded picture signal are again 
encoded the re-encoding is a function of the picture type 
included in the decoded picture signal. The re-encoded 
signal includes the picture type. Subsequent decoding of the lO 
re-encoded picture is a function of the picture type. Each 
picture of the re-decoded picture signal includes its respec- 
tive picture type. 

Thus, the present invention matches the type of predictive 
coding applied to pictures in a picture signal by serially 
arranged coders which process the picture signal. 

The present invention promotes optimum picture quality. 
A picture, previously encoded as an I-picture, P-picture or 
B-picture is again encoded as an I-picture, P-picture or 
B-picture, respectively. Also, encoding of a picture, previ- 
ously encoded as a B-picture, as an I-picture or a P-picture 
is prevented. Thus, deterioration in signal quality after plural 
coding and decoding operations is minimized. 

Referring now to the drawings, and in particular to FIG. 
14, there are illustrated coding and decoding units (codecs) 
according to the present invention having a serial analog 
connection therebetween. A first codec comprises coder 120 
and decoder 121, while a second codec conqmses coder 122 
and decoder 123. It will be appreciated by one of ordinary 
sJdii in the art that additional codecs may be serially con- 
nected to those shown in FIG. 14. 

In FIG. 14, an analog video signal is supplied to an input 
terminal 100 as an input signal a, and a picture type s^nal 
is supplied to an input terminal 108. The picture type 
indicates one of intra-picture coding (an I-picture), predic- 
tive coding (a P-picture) and bi-directionally predictive 
coding (a B-picture). 

The input terminals 100, 108 function to apply the analog 
video signal and the picture type signal, respectively, to an 
A/D converter 101 and a coding circuit 102, respectively, of 
coder 120. The converter 101 is adapted to convert the 
analog video signal to a digital video signal, and to apply the 
digital video signal to the coding circuit 102. 

The coding circuit 102 serves to encode the digital video 45 
signal as a function of the picture type signal to produce a 
coded digital video signal which includes, for each encoded 
picture, its picture type as identified by the picture type 
signal. More specifically, if the picture type for a picture 
indicates intra-picture coding, then the coding circuit 102 50 
codes the picture as an I-picture. If the picture type for a 
picture indicates predictive coding, then the coding circuit 
102 codes the picture as a P-picture. J£ the picture type for 
a picture indicates bi-directionaUy predictive coding, then 
the coding circuit 102 codes the picture as a B-picture. 55 

The coding unit 120 may alternatively have the structure 
shown in FIG. 6, in which coding is performed without 
reference to an externally supplied picture type. As used 
herein, an externally supplied picture type means a picture 
type supplied from generally the same source as supplies the 60 
digital video signal, rather ttim from a separate source as is 
shown in FIG. 6, namely, the picture type input device 65. 

The coded digital video signal from coding circuit 102 of 
FIG. 14 is supplied to a decoding circuit 103 of decoder 121 
which is adapted to decode the coded digital video signal as 65 
a function of the picture type included in the encoded signal 
and to apply the decoded video signal to D/A convaler 104. 
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llie decoding circuit 103 is further adapted to apply the 
picture type decoded from the coded digital video signal to 
a multiplexer 105. 

The multiplexer 105 is operative to multiplex the picture 
5 type information with the decoded video signal to produce 
a multiplexed analog video signal as an output signal b in 
which the picture type information is contained in the 
decoded video signal. Preferably, the multiplexer 105 inserts 
the picture type for a picture of the decoded video signal into 
10 the vertical blanking interval of the picture. As mentioned, 
a picture may be either a frame or a field of the video signal. 

The output signal b is supplied from the multiplexer 105 
to a separating circuit 106 of the cod^ 122. The separating 
circuit is operative to sq)arate or demultiplex the analog 
video signal and the picture type information from the 
output signal b, to supply the separated analog video signal 
to an A/D converter 107, and to supply the separated picture 
type information to a coding circuit 108. The converter 107 
is adapted to convert the separated analog video signal to a 
digital video signal, and to ^ply the digital video signal to 
the coding circuit 108. 

The coding circuit 108 serves to encode the digital video 
signal as a function of the separated picture type to produce 
25 a re-coded digital video signal which includes, for each 
re-encoded picture, its picture type as identified by the 
separated picture type signal. 

The re-coded digital video signal from the coding circuit 
108 is supplied to the decoder 123, which operates in a 
30 similar manner as the decoder 121. 

Decoding circuit 110 of decoder 123 decodes the re-coded 
digital video signal to produce a re-decoded digital video 
signal and a corresponding picture type signal. The 
re-decoded digital video signal is converted to an analog 

35 signal by D/A converter 109, and applied to a multiplexer 
111 which multiplexes the analog video signal with the 
picture type signal from decoding circuit 110 to produce a 
multiplexed analog video signal as an output signal c. 
Preferably, the multiplexer 111 inserts the picture type for a 

40 picture into the vertical blanking interval of the picture. The 
multiplexer 111 applies its ou^ut signal c to an output 
terminal 119. 

Due to the inclusion of the picture type identifier in the 
signals b and c, the codecs of FIG. 14 process respective 
45 pictures of the video signals b and c in the same manner, that 
is, as the same one of an I-picture, a P-picture or a B-picture. 

FIG. 15 shows the SNR of the output signals b, c shown 
in FIG. 14. The SNR of the output signal c is seen to be only 
slightly worse than the SNR of the output signal b. 

That is, since the type of predictive coding applied to each 
picture is the same in each of the serially arrai^ed codecs, 
the deterioration in picture quality at each codec is mini- 
mized even when the picture quality changes from picture to 
55 picture due to the type of predictive coding employed from 
picture to picture. 

FIG. 16 shows codecs according to the present invention 
having a serial d^tal connection therebetween. A first codec 
comprises coder 142 and decoder 143, while a second codec 
60 comprises coder 144 and decoder 145. 

In FIG. 16, an analog video signal is suppUed to an input 
terminal 140 as an input signal a, and a picture type signal 
is supplied to an input temtiinal 148. The input teaminals 140, 
148 function to apply the analog video signal and the picture 
65 type signal, respectively, to an A/D converter 141 and a 
coding circuit 152 of coder 142, respectively. The converter 
141 is adapted to convert the analog video signal to a digital 
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video signal, and to apply the digital video signal to a digital 
interface 151 of coder 142. 

The coding circuit 152 serves to encode the digital video 
signal as a function of the picture type signal to produce a 
coded digital video signal which includes, for each encoded ^ 
picture, its picture type as identified by the picture type 
signal. More specifically, if the picture type for a picture 
indicates intra-picture coding, then the coding circuit 152 
codes the picture as an I-picture. If the picture type for a 
picture indicates predictive coding, then the coding circuit 
152 codes the picture as a P-picture. If the picture type for 
a picture indicates bi-directionally predictive coding, then 
the coding circuit 152 codes the picture as a B-picture. 

The coding unit 120 may alternatively have the structure 
shown in FIG. 6, in which coding is performed without 
reference to an externally supplied picture type. 

The coded digital video signal firom coding circuit 152 is 
supplied to a decoding circuit 153 of decoder 143 which is 
adapted to decode the coded digital video signal as a 
function of the picture type included in the encoded signal 
and to apply the decoded video signal to a digital interface 
154. The decoding circuit 153 is further adapted to apply the 
picture type decoded firom the coded digital video signal to 
a multiplexer 155. ^5 

The multiplexer 155 is operative to multiplex the picture 
type information with the decoded video signal to produce 
a multiplexed digital video signal as an output signal b in 
which the picture type information is contained in tiie 
decoded video signal. Preferably, multiplexer 155 multi- 3Q 
plexes the picture type for a picture of the decoded video 
signal as a flag in the respective picture. 

In a prefoxed embodiment of the present invention, the 
picture type information is inserted (or multiplexed) into the 
decoded video signal at a location therein whidi precedes 35 
the actual video data that represents the video field or frame. 
FIGS. 17A, 17B and 17C illustrate the data structure of the 
decoded (MPEG) video signal. FIG. 17A illustrates the data 
structure of a decoded video signal having a serial digital 
interface format as specified in the standard SMFTE — ^259 40 
(Society of Motion Picture & Television Engineers). As 
shown, a frame consists of a first vertical blanking area 
VBKl, a first optional blanking area OBKl and a first active 
video area ACVl, which constitutes the first field, followed 
by a second vatical blanking area VBK2, a second optional 45 
blanking area OBK2 and a second active video area ACV2, 
which constitutes the second field. In the preferred 
embodiment, each of tiie vertical blanking areas consists of 
9 horizontal scanning lines, each of the optional bla nkin g 
areas consists of 10 horizontal scanning lines, the first active 50 
video area consists of 244 horizontal scanning lines and the 
second active video area consists of 243 horizontal scanning 
lines, for a total of 525 horizontal scanning lines for a single 
firame. 

HG. 17B Uiustrates the data structure of a horizontal 55 
scanning line. As shown, a horizontal scanning line includes 
an end of active video (EAV) area followed by an ancillary 
(ANC) area, a start of active video (SAV) area and a video 
area. The ANC area, as shown in FIG. 17C, includes an 
ancillary data flag (ADF) area followed by a data identtfi- 60 
cation (DID) area, an ancillary number data (DBN) area, an 
ancillary word data (DC) area, an ancillary data (ANC 
DATA) area, and a check sum (CS) area. Since the above- 
noted areas of a horizontal scanning line of digital data are 
weU-known in the art, their descriptions are omitted herein 65 
except where necessary for an understanding of the present 
invention. 
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In accordance with the present invention, the picture type 
information is insated into the DID area of the ANC area of 
each of the horizontal scanning lines in the first and second 
vertical blanking areas. However, the picture type informa- 
5 tion also may be inserted into the DID area of tiie ANC area 
of other horizontal scanning lines of the decoded video 
signal, although these DID areas may be used for transmit- 
ting other types of data. For example, the DID areas in 
horizontal lines of non-vertical blanMng areas may include 
other formatting information. 

The picture type information (or picture type data) may 
identify the type of encoding of the picture (e.g., intra- 
picture coding, predictive coding, and bi-directionaUy pre- 
dictive coding) in various ways. FIGS. 18A to 18C illustrate 
one method in which the type of encoding is identified by the 
structure of the group of pictures (GOP). As shown, the 
structure of a group of pictures (GOP) may be identified by 
the minimum number of frames "M" between I and P 
pictures, between P and P pictures, and between I and I 
pictures, and fee total number of frames **N'* (pictures) in the 
group of pictures. For exan^le, FIG. ISA illustrates groups 
of pictures having an encoding structure of M=3 and N=9 in 
which th^e are 9 frames in each group and wherein there are 
3 frames from each I or P frame to the respectively suc- 
ceeding I or P frame. Similarly, FIG. 18B illustrates groups 
of pictures having an encoding structure of M=2 and N=2, 
and FIG. 18C illustrates groups of pictures having an 
encoding structure of M=l and N=2. 

When the picture type data identifies the ^'M" and "ISP' 
numbers, the type of encoding for each picture can be 
determined by the location of a respective picture within the 
group of pictures, and the location of a respective picture 
may be identified in the picture type data either by identi- 
fying each picture's location within the group of pictures or 
by identifying only the first picture within the group of 
pictures. For example, when M=3 and N=9 for a group of 
frames in the decoded video signal (FIG. 18A), the third 
frame in that group is identified as a decoded I-frame. 
Returning to FIG. 16, the output signal b (the multiplexed 
^ decoded video signal) is supplied from the multiplexer 155 
to a separatii^ circuit 156 of the coder 144. The separating 
circuit is operative to separate or demultiplex the digital 
video signal and the picture type data from the output signal 
b, to supply the separated digital video signal to a digital 
interface 157, and to supply the separated picture type data 
to a coding circuit 158. The interface 157 is adapted to apply 
the separated digital video signal to the coding circuit 158. 

The coding circuit 158 serves to encode the separated 
digital video signal as a function of the separated picture 
5Q type to produce a re-coded digital video signal which 
includes, for each re^encoded picture, its picture type as 
identified by the sq>arated picture type signal. 

The re-coded digital video signal from the coding circuit 
158 is supplied to the decoder 145, which operates in a 
55 similar manner as the decoder 143. 

Decoding circuit 160 of decoder 145 decodes the re-coded 
digital video signal to produce a re-decoded digital video 
signal and a corresponding picture type signal. The 
re-decoded digital video signal is supplied to a digital 
60 interface 159 and thence to a multiplexer 161 which multi- 
plexes the re-decoded digital video signal with the picture 
type signal from decoding circuit 160 to produce a multi- 
plexed digital video signal. The multiplexer 161 multiplexes 
the picture type signal as a flag (e.g., in the DID area of the 
55 ANC area) in the re-decoded digital video signal. 

The multiplexer 161 supplies the multiplexed digital 
video signal to an A/D converter 146 which serves to convert 
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the mult^lexed digital video signal to an analog video signal 
also referred to as output signal c. The converter 146 iq>plies 
the output signal c to an output terminal 147. 

Due to the inclusion of the picture type identifier in the 
signals b and c, the codecs of FIG, 16 process respective 5 
pictures of tiie video signals b and c in the same mannea:, that 
is, as the same one of an I-picture, a P-picture or a B-picture, 
Consequently, the deterioration in pictoe quality at each 
codec is minimized even when the picture quality changes 
from picture to picture due to the type of predictive coding 
employed from picture to picture. 

FIG. 19 shows the coding circuits of HGS. 14 and 16 in 
more detail. In FIG. 19, elements similar to those in FIG. 6 
are indicated by the same reference numerals, and detailed 
explanations thereof are omitted. 

In FIG. 19, a picture type signal is supplied to input 
terminal 70 which serves to supply the picture type signal to 
motion vector detector 450, predictive judging circuit 454 
and variable length coding circuit 458. The processing 
performed by elements 450, 454 and 458 is similar to the 
processing performed by elements 50, 54 and 58 of FIG. 6, ^ 
except that the elements of FIG. 19 perform in accordance 
with the picture type identified in the external picture type 
signal which indicates the picture type used in previous 
coding. The variable length coding circuit 458 includes the 
picture type based on the external picture type signal as part 25 
of the header information. 

FIG. 20 shows the decoding circuits of FIGS. 14 and 16 
in more detail. In HG. 20, elements similar to those in FIG. 
9 are indicated by the same reference numerals, and detailed 
explanations thereof are omitted. 30 

Variable length decoding circuit 482 of FIG. 20 is similar 
to variable length decoding circuit 82 of FIG. 9, except that 
circuit 482 applies the picture type separated from the 
encoded signal not only to motion compensator 487, but also 
to ou^ut terminal 92, 

Although an illustrative embodiment of the present 
invention, and various modifications tiiereof, have been 
described in detail herein with reference to the accompany- 
ing drawings, it is to be understood that the invention is not 
Hmited to this precise embodiment and the described 40 
modifications, and tfiat various changes and further modi- 
fications may be effected therein by one skilled in the art 
without departing from the scope or spirit of the invention as 
defined in the appended claims. 

What is claimed is: 

1. An ^paratus for processing a digital picture signal, 
comprising: 

means for receiving a digital picture signal having picture 
type data included in a data identification area of said 
digital picture signal indicating one of intrapicture 50 
coding, predictive coding and bidirectionally predictive 
coding for respective pictures represented by said digi- 
tal picture signal, said picture type data identifying an 
encoding structure of a group of pictures represented by 
said digital picture signal and fiirther identifying each 55 
respective picture within said group of pictures so as to 
identify the type of encoding of said digital picture 
signal for each said picture; and 

coding means for encoding said digital picture signal as a 
function of said picture type data to produce an 60 
encoded digital picture signaL 

2. The apparatus of claim 1, wherein said picture type data 
identifies previous types of coding for said respective pic- 
tures represented by said digital picture signal. 

3. The apparatus of claim 1, wherein said means for 65 
receiving includes means for extracting said picture type 
data from said digital picture signal. 
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I 4. The apparatus of claim 1, wherein said picture type data 
; is included in a data identification area of at least a vertical 
blanking interval of said digital picture signal, 

5. The apparatus of claim 1, further comprising means for 
; 5 decoding said encoded digital picture signal as a function of 

said picture type data, 

6. The apparatus of claim 1, wherein said coding means 
includes motion vector detection means for detecting mcdon 
vectors between said pictures represented by said digital 
picture signal as a function of said picture type data: 
predictive judging means for choosing one of intra-coding, 
forward predictive coding, backward predictive coding and 
bi-directionally predictive coding said digital picture signal 
as a function of said picture type data; and variable length 
coding means for encoding said picture type data in said 
encoded digital picture signal. 

7. The apparatus of claim 1, wherein said digital picture 
signal includes an ancillary area in which said picture type 
data is included followed by a video area in which picture 
data representing a picture of said digital picture signal is 

20 included, 

8. The apparatus of claim 1, wherein said picture type data 
identifies a minimum number of frames between two firames 
encoded either by intrapicture or predictive coding and 

. identifies a total number of frames in said group of pictures 
25 represented by said d%ital picture signal, 

9. An apparatus for processing an encoded digital picture 
signal* comprising: 

means for decoding said encoded digital picture signal to 
produce picftire type data representing a type of encod- 

30 ing of said encoded digital picture signal and to pro- 
duce a decoded digital picture signal said picture type 
data identifying a previous encoding structure of a 
group of pictures represented by said encoded digital 
picture signal and further identifying eadi respective 

35 picture within said group of pictures rqpresented by 
said decoded digital picUire signal so as to identify the 
previous type of encoding for each picture rq)resented 
by said decoded digital picture signal; and 
means for including said picture type data in a data 

40 identification area of said decoded digital picture signal 
to produce an output signal, 

10. The j^paratus of claim 9, wherein said means for 
including is operable to include said picture type data in a 
data identification area of at least a vertical blanking interval 

45 of said decoded digital picture signal to produce said output 
signal. 

11. The ^paratus of claim 9, further conqaising means 
for encodii^ said decoded digital picture signal as a function 
of said picture type data 

50 12. ITie ^paratus of claim 9, wherein said means for 
decoding includes variable length decoding means for sepa- 
rating said picture type data from said encoded digital 
picture signal and wherein said means for including is 
operative to include the separated picture type data in said 

55 decoded digital picture signal, 

13, The apparatus of claim 9, wherein said digital picture 
signal includes an ancillary area in which said picture type 
data is included followed by a video area in which picture 
data representing a picture of said digital picture signal is 

60 included 

14, The apparatus of claim 9, wherein said picture type 
data identifies a minimum nuniber of frames between two 
frames encoded either by intrapicture or predictive coding 
and identifies a total numba: of frames in said group of 

65 pictures represented by said digital picture signal, 

15, A method of processing a digital picture signal, 
comprising the steps of: 
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receivicig a digital picture signal having picture type data 
included in a data identification area of said digital 
picture signal indicating one of intrapicture coding, 
predictive coding and bidirectionally predictive coding 
for respective pictures represented by said digital pic- 5 
ture signat said picture type data identifying an encod- 
ing structure of a group of pictures represented by said 
digital picture signal and further identifying each 
respective picture wi&in said group of pictures so as to 
identify the type of encoding of said d^ital picture lO 
signal for each said picture; and 

encoding said digital picture signal as a function of said 
picture type data to produce an encoded digital picture 
signal. 

16. The method of claim 15, wherein said picture type 
data identifies previous types of coding for said respective 
pictures represented by said digital picture signal. 

17. The method of claim 15, wherein said step of receiv- 
ing is carried out by extracting said picture type data from 
said digital picture signaL 20 

18. The method of claim 15, wherein said picture type 
data is included in a data identification area of at least a 
vertical blanking interval of said digital picture signal. 

19. The method of claim 15, further comprising the step 

of decoding said encoded digital picture signal as a function 25 
of said picture type data. 

20. Hie mediod of claim 15, wherein said step of encod- 
ing is carried out by detecting motion vectors between said 
pictures represented by said digital picture signal as a 
function of said picture type data; choosing one of intra- 3^ 
coding, forward predictive codmg, backward predictive cod- 
ing and bi-directionally predictive coding said digital picture 
signal as a function of said pictoe type data; and variable 
length encoding said picture type data in said encoded 
digital picture signal. ^5 

21. The method of claim 15, wherein said digital picture 
signal includes an ancillary area in which said picture type 
data is included followed by a video area in which picture 
data representing a picture of said digital picture signal is 
included. 
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22. The method of claim 15, wherein said picture type 
data identifies a minimum number of frames between two 
frames encoded either by intrapicture or predictive coding 
and identifies a total number of frames in said group of 

5 pictures represented by said digital picture signal. 

23. A mediod of processing an encoded digital picture 
signal, comprising the steps of: 

decoding said encoded digital picture signal to produce 
picture type data representing a type of encoding of 
said encoded digital picture signal and to produce a 
decoded digital picture signal, said picture type data 
identifying a previous encoding structure of a group of 
pictures represented by said encoded digital picture 
signal and further identifying eadi respective picture 
within said group of pictures represented by said 
decoded digital picture s^nal so as to identify the 
previous type of encoding for each picture rq)resented 
by said decoded digital picture signal; and 
including said picture type data in a data identification 
area of said decoded digital picture signal to produce an 
20 output signal. 

24. The apparatus of claim 23, wherein said step of 
including is carried out by including said picture type data 
in a data identification area of at least a vertical blanking 
interval of said decoded digital picture signal to produce said 

25 output signal. 

25. The method of claim 23, further comprising the step 
of encoding said decoded digital picture signal as a function 
of said picture type data, 

26. The method of claim 23, wherein said digital picture 
30 signal includes an ancillary area in which said picture type 

data is included followed by a video area in which picture 
data representing a picture of said digital picture signal is 
included, 

27. The method of claim 23, wherein said picture type 
35 data identifies a minimum number of frames between two 

frames encoded either by intrapicture or predictive coding 
and identifies a total number of frames in said group of 
pictures rq)resented by said digital picture signal. 
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2 8 . An encodiriQ apparatus for encoding source video 
data which had previously been encoded at a previous encoding 
process and had previously been decoded at a previous decoding 
process, said apparatus comprising: 

means for receiving said source video data; 
means for extracting coding information from said 
source video data, wherein said coding information relates to a 
coding operation of said previous encoding process; and 

means for encoding said source video data in accordance 
with said coding information. 
y 29 . An encoding method for encoding source video data 

which had previously been encoded at a previous encoding process 
and had previously been decoded at a previous decoding process, 
the method comprising the steps of: 

receiving said source video data; 
y extracting coding information from said source video 

£^ data, wherein said coding information relates to a coding 
O operation of said previous encoding process; and 

encoding said source video data in accordance with said 
coding information. 

30 . An encoding apparatus for encoding source video 
data, said apparatus comprising: 

means for receiving said source video data, 
wherein said source video data had previously been encoded at a 
previous encoding process, and for receiving coding information 
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relating to a coding operation of said previous encoding process; 
and 

means for encoding said source video data in accordance 
with said coding information. 

31 . An encoding method for encoding source video data, 
the method comprising the steps of: 

receiving said source video data, wherein said 
source video data had previously been encoded at a previous 
encoding process, and for receiving coding information relating 
to a coding operation of said previous encoding process; and 

encoding said source video data in accordance with 
said coding information. 

32 . An encoding apparatus for encoding source video 
data, said apparatus comprising: 

means for receiving a plurality of pictures within 
said source video data, wherein said plurality of pictures had 
previously been encoded at a previous encoding process; 

means for receiving picture coding type indicating 
which of 1-picture, P-picture or B-picture had been associated 
with said previous encoding process; and 

means for encoding each of said pictures so that 
each picture is encoded by using the same picture coding type as 
said picture coding type of said previous encoding process. 

33 . An encoding method for encoding source video data, 
the method comprising the steps of: 
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receiving a plurality of pictures within said 
source video data, wherein said plurality of pictures had 
previously been encoded at a previous encoding process; 

receiving picture coding type indicating which of 
1-picture. P-picture or B-picture had been associated with said 
previous encoding process; and 

encoding each of said pictures so that each 
picture is encoded by using the same picture coding type as said 
picture coding type of said previous encoding process, 

34 . A decoding apparatus for decoding an encoded bit 
stream which had been encoded at a previous encoding process, 
said apparatus comprising: 

means for decoding said encoded bit stream to 
generate decoded video data in accordance with coding information 
relating to a coding operation of said previous encoding process; 

means for multiplexing said decoded video data and 
said coding information to generate multiplexed data; and 

means for transmitting said multiplexed data so 
that said coding information will be used in a later encoding 
process . 

3 5 . A decoding method for decoding an encoded bit 
stream which had been encoded at a previous encoding process, the 
method comprising the steps of: 

decoding said encoded bit stream to generate 
decoded video data in accordance with coding informat ion relating 
to a coding operation of said previous encoding process; 
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multiplexing said decoded video data and said 
coding information to Qenerate multiplexed data; and 

transmitting said multiplexed data so that said 
coding information will be used in a later encoding process. 

36 . A decoding apparatus for decoding an encoded bit 
stream which had been encoded at a previous encoding process, 
said apparatus comprising: 

means for decoding said encoded bit stream to 
generate decoded video data; 

means for multiplexing said decoded video data and 
coding information relating to a coding operation of said 
previous encoding process; and 

means for transmitting the multiplexed data so 
that said coding information will be used in a later encoding 
process , 

^ 3 7. A decoding method for decoding an encoded bit 

stream which had been encoded at a previous encoding process, the 
method comprising the steps of: 

decoding said encoded bit stream to generate 
decoded video data; 

multiplexing said decoded video data and coding 
information relating to a coding operation of said previous 
encoding process; and 

transmitting the multiplexed data so that said 
coding information will be used in a later encoding process. 
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3 8 , A decoding apparatus for decoding an encoded bit 
stream which had been encoded at a previous encoding process, 
said apparatus comprising: 

means for extracting coding information from said 
encoded bit stream, wherein said coding information relates to a 
coding operation of said previous encoding process; 

means for decoding said encoded bit stream to 
generate decoded video data in accordance with said coding 
information; and 

means for transmitting said decoded video data and 
Q said coding information so that said coding information will be 

used in a later encoding process for said decoded video data. 
n 3 9 . A decoding method for decoding an encoded bit 

^ stream which had been encoded at a previous encoding process, the 
^ " method comprising the steps of: 

M extracting coding information from said encoded 

0 bit stream, wherein said coding information relates to a coding 
p operation of said previous encoding process; 

decoding said encoded bit stream to generate 

decoded video data in accordance with said coding information; 

and 

transmitting said decoded video data and said 
coding information so that said coding information will be used 
in a later encoding process for said decoded video data. 
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40 . A decoding apparatus for decoding an encoded bit 
stream which had been encoded at a previous encoding process, 
said apparatus comprising: 

means for extracting coding information from said 
encoded bit stream, wherein said coding information relates to a 
coding operation of said previous encoding process; 

means for decoding said encoded bit stream to 
generate decoded video data; and 

means for transmitting the decoded video data and 
said coding information so that said coding information will be 
n used in a later encoding process for said decoded video data. 

41 . A decoding method for decoding an encoded bit 

?1 stream which had been encoded at a previous encoding process, the 

"it method comprising the steps of: 

extracting coding information from said encoded 

y bit stream, wherein said coding information relates to a coding 

O operation of said previous encoding process; 

O decoding said encoded bit stream to generate 

decoded video data; and 

transmitting the decoded video data and said 
coding information so that said coding information will be used 
in a later encoding process for said decoded video data. 

42 . A decoding apparatus for decoding an encoded bit 
stream which had been encoded at a previous encoding process, 
said apparatus comprising: 
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means for extracting picture coding type from said 
encoded bit stream, wherein said picture coding type indicates 
which of I -picture. P-picture, or B-Picture had been associated 
with said previous encoding process; 

means for decoding each picture within said 
encoded bit stream to generate decoded video data; and 

means for transmitting said decoded video data and 
said picture coding type so that each said picture will be 
encoded by using the same picture coding type as said picture 
coding type in a later encoding process for said decoded video 
data . 

43 . A decoding method for decoding an encoded bit 
stream which had been encoded at a previous encoding process, the 
method comprising the steps of: 

extracting picture coding type from said encoded 
bit stream, wherein said picture coding type indicates which of 
1 -picture, P-picture, or B-Picture had been associated with said 
previous encoding process; 

decoding each picture within said encoded bit 
stream to generate decoded video data; and 

transmitting said decoded video data and said 
picture coding type so that each said picture will be encoded by 
using the same picture coding type as said picture coding type in 
a later encoding process for said decoded video data. 
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44 . A coding system for performing a decoding lorocess 
and an encoding process to an encoded bit stream which had been 
encoded at a previous encoding process, the system comprising: 

decoding means for decoding said encoded bit 
stream to generate decoded video data, and for outputting coding 
information relating to a coding operation of said previous 
encoding process; and 

encoding means for encoding said decoded video 
data based on said coding information transmitted from said 
decoding means . 

4 5 . A coding method for performing a decoding process 
and an encoding process to an encoded bit stream which had been 
encoded at a previous encoding process, the method comprising the 
steps of: 

decoding said encoded bit stream by use of a 
decoder to generate decoded video data and outputting coding 
information relating to a coding operation of said previous 
encoding process; and 

encoding said decoded video data based on said 
coding information transmitted from said decoder. 

4 6 . A coding system for performing a decoding process 
and an encoding process to an encoded bit stream which had been 
encoded at a previous encoding process, the system comprising: 

decoding means for decoding said encoded bit 
stream to generate decoded video data; 
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encoding means for encoding said decoded video 

data; and 

means for controlling a coding operation of said 
encoding means in accordance with coding information relating to 
a coding operation of said previous encoding process. 

4 7 . A coding method for performing a decoding process 
and an encoding process to an encoded bit stream which had been 
encoded at a previous encoding process, the method comprising the 
steps of: 

decoding said encoded bit stream to generate 
decoded video data; 

encoding said decoded video data by use of an 

^ encoder; and 

„^ ^ I 

Jr! controlling a coding operation of said encoder in 

accordance with coding information relating to a coding operation 
O of said previous encoding process. 
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ABSTRACT 



A picture type identiiiear, indicating one of intra-picture 
coding (an I-picture), forward or backward predictive cod- 
ing (a P-picture) and bi-directionally predictive coding (a 
B-picture), is included with a picture signal when the signal 
is encoded and when the signal is decoded. Each of initial 
and subsequent encoding and decoding is a function of the 
included picture type. 
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As a below-named inventor, I hereby declare that: 

1. My residence, post office address and citizenship 
are as stated below next to my name, 

2. I verily believe myself to be the original, first 
inventor of the invention described and claimed in Letters Patent 
No. 5,715,009 and in the specification filed herewith for which I 
solicit a patent. 

3 . I hereby state that I have reviewed and understand 
the contents of the aforementioned specification, including the 
claims . 

4 . I acknowledge the duty to disclose information 
which is material to the examination of this application in 
accordance with Title 37, Code of Federal Regulations, § 1.56(a). 

5 . I hereby claim foreign priority benefits under 
Title 35, United States Code § 119 of Application Number 06- 
130651 filed in Japan on June 13, 1994 and state that no other 
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international application was filed by me on the same subject 
matter prior to June 13, 1994, 

6, I do not know and do not believe that the invention 
was ever known or used in the United States of America before my 
invention thereof. 

7. I verily believe the original Letters Patent to be 
wholly or partly inoperative or invalid by reason of my claiming 
more or less than I had a right to claim in the patent by at 
least failing to claim the following: 

A. An encoding apparatus for encoding source video 
data which had previously been encoded at a previous encoding 
process and had previously been decoded at a previous decoding 
process, said apparatus comprising means for receiving said 
source video data; means for extracting coding information from 
said source video data, wherein said coding information relates 
to a coding operation of said previous encoding process; and 
means for encoding said source video data in accordance with said 
coding information . 

B. An encoding method for encoding source video data 
which had previously been encoded at a previous encoding process 
and had previously been decoded at a previous decoding process, 
the method comprising the steps of receiving said source video 
data; extracting coding information from said source video data, 
wherein said coding information relates to a coding operation of 
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said previous encoding process; and encoding said source video 
data in accordance with said coding information. 

C. An encoding apparatus for encoding source video 
data, said apparatus comprising means for receiving said source 
video data, wherein said source video data had previously been 
encoded at a previous encoding process, and for receiving coding 
information relating to a coding operation of said previous 
encoding process; and means for encoding said source video data 
in accordance with said coding information. 

D. An encoding method for encoding source video data, 
the method comprising the steps of receiving said source video 
data, wherein said source video data had previously been encoded 
at a previous encoding process, and for receiving coding 
information relating to a coding operation of said previous 
encoding process; and encoding said source video data in 
accordance with said coding information. 

E. An encoding apparatus for encoding source video 
data, said apparatus comprising means for receiving a plurality 
of pictures within said source video data, wherein said plurality 
of pictures had previously been encoded at a previous encoding 
process; means for receiving picture coding type indicating which 
of I-picture, P-picture or B-picture had been associated with 
said previous encoding process; and means for encoding each of 
said pictures so that each picture is encoded by using the same 
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picture coding type as said picture coding type of said previous 
encoding process. 

F. An encoding method for encoding source video data, 
the method comprising the steps of receiving a plurality of 
pictures within said source video data, wherein said plurality of 
pictures had previously been encoded at a previous encoding 
process; receiving picture coding type indicating which of 1- 
picture, P-picture or B-picture had been associated with said 
previous encoding process; and encoding each of said pictures so 
that each picture is encoded by using the same picture coding 
type as said picture coding type of said previous encoding 
process . 

G. A decoding apparatus for decoding an encoded bit 
stream which had been encoded at a previous encoding process, 
said apparatus comprising means for decoding said encoded bit 
stream to generate decoded video data in accordance with coding 
information relating to a coding operation of said previous 
encoding process; means for multiplexing said decoded video data 
and said coding information to generate multiplexed data; and 
means for transmitting said multiplexed data so that said coding 
information will be used in a later encoding process. 

H. A decoding method for decoding an encoded bit 
stream which had been encoded at a previous encoding process, the 
method comprising the steps of decoding said encoded bit stream 
to generate decoded video data in accordance with coding 
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information relating to a coding operation of said previous 
encoding process; multiplexing said decoded video data and said 
coding information to generate multiplexed data; and transmitting 
said multiplexed data so that said coding information will be 
used in a later encoding process. 

1 . A decoding apparatus for decoding an encoded bit 
stream which had been encoded at a previous encoding process, 
said apparatus comprising means for decoding said encoded bit 
stream to generate decoded video data; means for multiplexing 
said decoded video data and coding information relating to a 
coding operation of said previous encoding process; and means for 
transmitting the multiplexed data so that said coding information 
will be used in a later encoding process. 

J. A decoding method for decoding an encoded bit 
stream which had been encoded at a previous encoding process, the 
method comprising the steps of decoding said encoded bit stream 
to generate decoded video data; multiplexing said decoded video 
data and coding information relating to a coding operation of 
said previous encoding process; and transmitting the multiplexed 
data so that said coding information will be used in a later 
encoding process. 

K. A decoding apparatus for decoding an encoded bit 
stream which had been encoded at a previous encoding process^ 
said apparatus comprising means for extracting coding information 
from said encoded bit stream, wherein said coding information 
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relates to a coding operation of said previous encoding process; 
means for decoding said encoded bit stream to generate decoded 
video data in accordance with said coding information; and means 
for transmitting said decoded video data and said coding 
information so that said coding information will be used in a 
later encoding process for said decoded video data. 

L. A decoding method for decoding an encoded bit 
stream which had been encoded at a previous encoding process, the 
method comprising the steps of extracting coding information from 
said encoded bit stream, wherein said coding information relates 
to a coding operation of said previous encoding process; decoding 
said encoded bit stream to generate decoded video data in 
accordance with said coding information; and transmitting said 
decoded video data and said coding information so that said 
coding information will be used in a later encoding process for 
said decoded video data. 

M. A decoding apparatus for decoding an encoded bit 
stream which had been encoded at a previous encoding process, 
said apparatus comprising means for extracting coding information 
from said encoded bit stream, wherein said coding information 
relates to a coding operation of said previous encoding process; 
means for decoding said encoded bit stream to generate decoded 
video data; and means for transmitting the decoded video data and 
said coding information so that said coding information will be 
used in a later encoding process for said decoded video data. 
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N. A decoding method for decoding an encoded bit 
stream which had been encoded at a previous encoding process, the 
method comprising the steps of extracting coding information from 
said encoded bit stream, wherein said coding information relates 
to a coding operation of said previous encoding process; decoding 
said encoded bit stream to generate decoded video data; and 
transmitting the decoded video data and said coding information 
so that said coding information will be used in a later encoding 
process for said decoded video data. 

0. A decoding apparatus for decoding an encoded bit 
stream which had been encoded at a previous encoding process, 
said apparatus comprising means for extracting picture coding 
type from said encoded bit stream, wherein said picture coding 
type indicates which of I-picture, P-picture, or B-Picture had 
been associated with said previous encoding process; means for 
decoding each picture within said encoded bit stream to generate 
decoded video data; and means for transmitting said decoded video 
data and said picture coding type so that each said picture will 
be encoded by using the same picture coding type as said picture 
coding type in a later encoding process for said decoded video 
data. 

P. A decoding method for decoding an encoded bit 
stream which had been encoded at a previous encoding process, the 
method comprising the steps of extracting picture coding type 
from said encoded bit stream, wherein said picture coding type 
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indicates which of I -picture, P-picture, or B-Picture had been 
associated with said previous encoding process; decoding each 
picture within said encoded bit stream to generate decoded video 
data; and transmitting said decoded video data and said picture 
coding type so that each said picture will be encoded by using 
the same picture coding type as said picture coding type in a 
later encoding process for said decoded video data. 

Q. A coding system for performing a decoding process 
and an encoding process to an encoded bit stream which had been 
encoded at a previous encoding process, the system comprising 
decoding means for decoding said encoded bit stream to generate 
decoded video data, and for outputting coding information 
relating to a coding operation of said previous encoding process; 
and encoding means for encoding said decoded video data based on 
said coding information transmitted from said decoding means. 

R. A coding method for performing a decoding process 
and an encoding process to an encoded bit stream which had been 
encoded at a previous encoding process, the method comprising the 
steps of decoding said encoded bit stream by use of a decoder to 
generate decoded video data and outputting coding information 
relating to a coding operation of said previous encoding process; 
and encoding said decoded video data based on said coding 
information transmitted from said decoder. 

S. A coding system for performing a decoding process 
and an encoding process to an encoded bit stream which had been 
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encoded at a previous encoding process, the system comprising 
decoding means for decoding said encoded bit stream to generate 
decoded video data; encoding means for encoding said decoded 
video data; and means for controlling a coding operation of said 
encoding means in accordance with coding information relating to 
a coding operation of said previous encoding process. 

T, A coding method for performing a decoding process 
and an encoding process to an encoded bit stream which had been 
encoded at a previous encoding process, the method comprising the 
steps of decoding said encoded bit stream to generate decoded 
video data; encoding said decoded video data by use of an 
encoder; and controlling a coding operation of said encoder in 
accordance with coding information relating to a coding operation 
of said previous encoding process, 

8 . I did not discover that the claims of the original 
patent claimed more or less than 1 had a right to claim until 
after the original patent was issued. 

9. No claims were previously presented during 
prosecution of the above referenced issued patent that 
particularly claimed the methods and apparatuses described in 
paragraph 7 . 

10. The error (s) noted above, as well as any other 
errors to be corrected herein, arose without any deceptive 
intention on my part. 
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11. New claims 28-47 submitted with this application 
particularly point out subject matter which I considered my 
invention and round out the scope of protection to which I am 
entitled. By the omission of such claims the original patent 
claims less than I had a right to claim. 

I hereby appoint William S. Frommer, Registration No. 

25,506, and Dennis M. Smid, Registration No. 34,930, of Frommer 

Lawrence & Haug LLP or their duly appointed associate, my 

attorneys, with full power of substitution and revocation, to 

prosecute this application, to make alterations and amendments 

therein, to file continuation and divisional applications 

thereof, to receive the Patent, and to transact all business in 

the Patent and Trademark Office and in the Courts in connection 

therewith, and specify that all communications about the 

application are to be directed to the following address: 

William S. Frommer, Esq. 

c/o Frommer Lawrence & Haug LLP 

745 Fifth Avenue 

New York, New York 10151 

Direct all telephone calls to: (212) 588-0800 to the attention 
of William S. Frommer, Esq. 

Wherefore I pray that I may be allowed to surrender the 

Letters Patent No. 5,715,009 granted February 3, 1998, whereof 

Sony Corporation, on whose behalf and with whose assent this 

application is made, is the sole owner, by Assignment, and that 

Letters Patent may be reissued to Sony Corporation for the same 

invention upon the attached specification. 
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I, the undersigned applicant, further declare that all 
statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be 
true; and further that these statements are made with the 
knowledge that willful false statements and the like so made are 
punishable by fine or imprisonment, or both, under Section 1001 
of Title 18 of the United States Code, and that such willful 
false statements may jeopardize the validity of the application 
or any patent issuing thereon. 



Kat sumi Tahara Dat e 

Residence: Kanagawa, Japan 
Citizenship : Japan 



Hiromi Yoshinari Date 
Residence: Kanagawa, Japan 
Citizenship : Japan 

Post Office Address of Inventors: 
Sony Corporation 
7-35 Kitashinagawa 6-chome 
Shinagawa-ku 
Tokyo, Japan 
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